# Encoder-Decoder Architecture

MrT5 is an efficient byte-level language model based on ByT5 improvements, reducing input sequence length by approximately 50% through dynamic token merging technology

Shuka v1 is a language model natively supporting Indian language audio understanding, combining a self-developed audio encoder with the Llama3-8B-Instruct decoder, enabling zero-shot multilingual question-answering tasks.

Pile-T5 Base is an encoder-decoder model trained on The Pile dataset using the T5x library, trained for 2 million steps with MLM objective, approximately 2 trillion tokens.

Pile-T5 XXL is an encoder-decoder model trained on The Pile dataset using the T5x library, employing a MLM objective similar to the original T5 model, trained for 2 million steps (approximately 2 trillion tokens).

MedICap is an encoder-decoder model for medical image captioning, which won the championship in the ImageCLEFmedical Caption 2023 challenge.

An encoder-decoder keyword generation model based on Google's Transformer architecture, supporting Polish and English, primarily used for extracting keywords from scientific paper abstracts.

This is an encoder-decoder model based on the BERT-mini architecture, specifically fine-tuned for the CNN/Dailymail dataset for text summarization tasks.

This is an automatic speech recognition encoder-decoder model trained on the MOZILLA-FOUNDATION/COMMON_VOICE_7_0 German dataset, combining the strengths of Wav2Vec2 and GPT2 architectures.

A Turkish paraphrase generation model based on the Bert2Bert architecture, used to generate sentences with the same meaning but different expressions.

An encoder-decoder model initialized with RoBERTa-Large, specifically designed for summarization tasks and fine-tuned on the CNN/DailyMail dataset.

An encoder-decoder model fine-tuned on the cc_news_es_titles dataset for Spanish text processing tasks